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CLAIMS 

1 . A method of arranging image data representing a motion picture 
sequence (10, B1 etc) within a memory sub-system (150, 210,220) in an image 

5 data processing system (200-220), the method comprising dynamically selecting 
(1 90) the arrangement of image data for successive pictures (I0 f B1 etc) of said 
sequence in said memory according to at least one of: measured characteristics 
of said image data, measured characteristics of the performance of said 
processing system, and known characteristics of subsequent processing of said 

io image data within said image processing system. 

2. A method as claimed in claim 1 , wherein said memory sub-system 
(150) includes an image data storage memory (220) constructed from paged 
memory. 

15 

3. A method as claimed in claim 1 or 2, wherein said memory sub- 
system (150) includes a processor cache memory (210) in addition to a main 
image data storage memory (220). 

20 4. A method as claimed in any preceding claim, wherein the step of 

selecting the arrangement of image data in storage memory (220) comprises 
selecting between a linear format (Fig.4), whereby image data is stored in 
memory on a line-by-line basis, and at least one kind of tiled format (Fig.7, Fig.9), 
whereby two-dimensional groups of pixels are grouped in memory (210, 220). 

25 

5. A method as claimed in claim 3 or 4, wherein where the memory 
sub-system (150) includes cache memory (210), said tiled format (Fig.7, Fig.9) is 
defined such that data for one tile corresponds to a whole number of cache 
blocks (CBn). 
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6. A method as claimed in any preceding claim, wherein the method 
comprises measuring one or more of the following as characteristics of the image 
data to influence the selection of the arrangement of data in memory (21 0, 220):- 

A. variability of motion vectors encoded within the received 

5 data; 

B. picture type (I, B, P); 

C. encoded data size per picture of the sequence; 

D. structure of a picture from which the image data is derived; 

E. structure employed for storing an image in image memory; 
10 p. constant characteristics of the stream obtained by decoding the 

sequence header of the image stream. 

7. A method as claimed in claim 6, wherein the method looks ahead 



in the motion picture sequence so as to measure said characteristics of the 
15 image data for a given portion of the sequence and select the memory 
arrangement prior to processing that portion. . 

8. A method as claimed in claim 6 or 7, wherein measured 
characteristics of the image data at one part of the sequence are used effectively 

20 to predict characteristics of a subsequent portion of the sequence, and the 
memory arrangement controlled according to measured characteristics of 
recently processed portions of the sequence. 

9. A method as claimed in claim 6, 7 pr 8, wherein the measurement 
25 of image data characteristics is averaged over a period of time. 

10. A method as claimed in any of claims 6 to 9, wherein where the 
variability of motion vectors is measured separately between vertical and 
horizontal planes, each having a different effect in the selection of the storage 

30 arrangement. 
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11 . A method as claimed in any preceding claim, wherein said method 
comprises measuring one or more of the following as characteristics of system 
performance to influence the selection of the arrangement of data in memory 
(210, 220):- 

5 G. data cache stall rates in the memory sub-system; 

H. processor utilisation; 

I. quality of service, or other such qualitative measurements that are 
perceptible to the end user of the content being processed; 

J. bandwidth of a link feeding data into or out of said image 

io processor. 

12. A method as claimed in claim 1 1 , wherein where said memory sub- 
system (150) includes cache memory (220), measurement of system 
performance includes measurement of data cache stall rates during image 

15 reconstruction. 

13. A method as claimed in claim or 12, wherein system performance 
is measured on a test basis using a sample of data, prior to processing the data. 



20 



14. A -method as claimed claim 11, 12 or 13, wherein system 
performance measured while processing a first part of the sequence is used in 
selecting the arrangement of memory for a subsequent part of the sequence. 

15. A method as claimed in any preceding claim, wherein the method 
25 comprises using knowledge of subsequent processing steps to influence the 

selection of the arrangement of data in memory (210, 220). 

16. A method as claimed in any preceding claim, wherein the selection 
of memory arrangement is implemented at least partly by changing parameters 

30 used by memory-accessing program code. 
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17. A method as claimed in any preceding claim, wherein the selection 
of memory arrangement is implemented at least partly by selecting different 
versions of code to be executed. 

5 18. A method of processing image data representing a motion picture 

sequence (Fig.4) within a memory sub-system (150) in an image data processing 
system, the memory sub-system (150) including processor cache memory (210) 
in addition to main image data storage memory (220), the method comprising 
selectively using cache-handling functions under program control (200), 

10 according to at least one of: measured characteristics of said image data and 
measured characteristics of the performance of said processing system, among 
those listed in any of claims 2 to 17. 

19. A method as claimed in claim 18, wherein a block allocation 
15 function, whereby a new cache-block is allocated and overwritten without pre- 
loading it from the main memory (220), is used selectively according to said 
measured characteristics. 

20. A method as claimed in claims 18 or 19, wherein, in addition, 
- 20 cache pre-fetching is activated selectively in accordance with the measured 

characteristics. 

21. A method as claimed in any of claims 1 to 17, combined with the 
method as claimed in any of claims 18 to 20. 

25 

22. An image data processing system, the processing system 
including a memory sub-system (150, 210, 220) and means (190, 200) for 
dynamically selecting the arrangement of image data for successive frames of a 
motion picture sequence (10, B1 etc) within said memory sub-system (150) 

30 according to at least one of: measured characteristics of said image data, 
measured characteristics of the performance of said processing system, and 
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known characteristics of subsequent processing of said image data within said 
image processing system. 

23. A system as claimed in claim 22, wherein the memory sub-system 
5 (150) includes an image data storage memory (220) constructed from paged 

memory. 

24. A system as claimed in claim 23, wherein said memory sub-system 
(150) includes a processor cache memory (210) in addition to a main image data 

10 storage memory (220). 

25. A system as claimed in any of claims 22 to 24, wherein the means 
(90, 160, 170, 192, 196, 194) for selecting the arrangement of image data in 
memory (220) is arranged for selecting between a linear format (Fig.4), whereby 

15 image data is stored in memory (220) on a line-by-line basis, and at least one 
kind of tiled format (Fig.7, Fig.9), whereby two-dimensional groups of pixels are 
grouped in memory. 

26. A system as claimed in claim 24 or 25, wherein where the memory 
20 - sub-system (1 50) includes cache memory (21 0), said tiled format is defined such 

that data for one tile corresponds to a whole number of cache blocks (CBn). 

27. A system as claimed in any of claims 22 to 26, wherein the 
selecting means (190) includes means (190) for measuring one or more of the 

25 following as characteristics of the image data to influence the selection of the 

arrangement of data in memory:- 

A. variability of motion vectors encoded within the received 

data; 

B. picture type (I, B, P); 

30 c. encoded data size per picture of the sequence; 

p structure of a picture from which the image data is derived; 

E structure employed for storing an image in image memory; 
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p constant characteristics of the stream obtained by decoding the 

sequence header of the image stream. 

28. A system as claimed in claim 27, wherein the selecting means 
(190) is arranged to look ahead in the motion picture sequence (Fig.3) so as to 
measure said characteristics of the image data for a given portion of the 
sequence and select the memory arrangement prior to processing that portion. 

29. A system as claimed in claim 27 or 28, wherein the selecting 
means (190) is arranged such that measured characteristics of the image data at 
one part of the sequence are used effectively to predict characteristics of a 
subsequent portion of the sequence, and the memory arrangement controlled 
according to measured characteristics of recently processed portions of the 
sequence. 

30. A system as claimed in claim 27, 28 or 29, wherein the measuring 
means (190) includes means for averaging measured image data characteristics 
over period of time. 

31 . A system as claimed in any of claims 27 to 30, wherein where the 
measuring means (190) is arranged to measure variability of motion vectors, it is 
arranged to do so separately between vertical and horizontal planes, each 
having a different effect in the selection of the storage arrangement. 

32. A system as claimed in of claims 22 to 31 , wherein the selecting 
means (190) alternatively or in addition comprises means (190) for measuring 
one or more of the following as characteristics of system performance to 
influence the selection of the arrangement of data in memory:- 

G. data cache stall rates in the memory sub-system; 

H. processor utilisation; 

I quality of service, or other such qualitative measurements that are 

perceptible to the end user of the content being processed; 



10 
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j bandwidth of a link feeding data into or out of said image 

processor, 

33. A system as claimed in 32, wherein where the memory sub-system 
5 (150) includes cache memory (220), the means for measuring system 

performance is arranged to measure data cache stall rates during image 
reconstruction. 

34. A system as claimed in claim 32 or 33, wherein the means for 
io measuring system performance is arranged to do so at least partly on a test 

basis using a sample of data, prior to processing the data. 

35. A system as claimed in claim 32, 33 or 34, wherein the selecting 
means and means for measuring system performance are arranged such that 

15 system performance measured while processing a first part of the sequence is 
used to influence the arrangement of memory for a subsequent part of the 
sequence. 

36. A system as claimed in any of claims 22 to 35, wherein the 
20 selecting means is arranged to use knowledge of subsequent processing steps 

to influence the selection of the arrangement of data in memory. 

37. A system as claimed in any of claims 22 to 36, wherein the image 
processing system is implemented at least in part by program code and a 

25 programmable processing unit (200). 

38. A system as claimed in any of claims 22 to 37, wherein the 
selecting means is implemented at least partly by means for changing 
parameters used by said program code in accessing said memory sub-system 

30 (150). 
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39. A system as claimed in any of claims 22 to 37, wherein the 
selecting means is implemented at least partly by selecting different versions of 
code to be executed. 

5 40. An image data processing system including a memory sub-system 

(150) for storing image data representing a motion picture sequence (Fig.3) 
being processed, the memory sub-system (150) including processor cache 
memory (210) in addition to main image data storage memory (220), the system 
being arranged for using cache-handling functions selectively under program 

10 control (200), according to at least one of measured characteristics of said 
image data and measured characteristics of the performance of said processing 
system, among those listed in any of claims 23 to 39. 

41 . A system as claimed in claim 40, wherein the system is arranged 
15 such that a block allocation function, whereby a new cache-block is allocated and 

overwritten without pre-loading it from the main memory (220), is selectively used 
according to said measured characteristics. 

42. A system as claimed in claims 40 or 41, wherein the system is 
20 _ arranged such that cache pre-fetching is activated selectively in accordance with 

the measured characteristics. 

43. A system as claimed in any of claims 22 to 39, combined with the 
system as claimed in any of claims 40 to 42. 

25 

44. A computer program product, whether on a physical storage 
device or delivered through some other channel, comprising instructions for 
causing a data processing system (200) to implement the steps of a method or 
the features of an apparatus embodying the invention as claimed in any 

30 preceding claim. 



